Multidimensional tensor processing

ABSTRACT

A tensor processing technique includes: accessing a tensor, wherein the tensor: represents interconnections of nodes across one or more dimensions, comprises a plurality of matrices, and forms a plurality of vectors across at least one of the one or more dimensions; applying Fourier Transform on the tensor to obtain a plurality of harmonic matrices; performing singular value decompositions (SVDs) on the plurality of harmonic matrices to obtain a plurality of corresponding SVD results; reducing the plurality of corresponding SVD results, including selecting one or more dominant components in the plurality of corresponding SVD results to obtain one or more reduced results; and performing Inverse Fourier Transform on the one or more reduced results to obtain a de-noised tensor.

CROSS REFERENCE TO OTHER APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/379,633 entitled MULTIDIMENSIONAL DATA PROCESSING filed Aug. 25,2016 which is incorporated herein by reference for all purposes.

BACKGROUND OF THE INVENTION

Cluster analysis (also referred to as clustering) is a technique forgrouping objects into groups (referred to a cluster) according tocertain criteria such that objects in the same group are more similar toeach other than those in other groups. Clustering is commonly used indata mining, statistical data analysis, machine learning, patternrecognition, and many other data processing applications. It issometimes used to pre-process data for further analysis.

Existing clustering techniques such as k-means typically representobjects in a two dimensional space and rely on search-and-eliminatecomputations to cluster data. These techniques often require multipleiterations and thus large amounts of processor cycles and/or memory,especially for processing massive amounts of data. Further, existingtechniques often rely on ad hoc approaches whose implementations areusually iterative and slow. The results are often limited in terms ofproviding insight into complex relationships among data points andeffectively measuring the influence of the clusters. Because theprocessing usually treats data sets independently, information about theinterconnections between different types of data is sometimes lost. Itwould be useful to have techniques that are more efficient and requireless computational resources. It would also be useful to have analyticalsolutions that are more easily parallelized, and that are able toprovide greater insight into the data relationships in multipledimensions.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 is a functional diagram illustrating a programmed computer systemfor performing data clustering in accordance with some embodiments.

FIG. 2 is a flowchart illustrating an embodiment of a clusteringprocess.

FIGS. 3A-3D are diagrams illustrating embodiments of graphs.

FIG. 4 is a diagram illustrating a visualization of a multi-dimensionaltensor example relating to airport delays.

FIG. 5 is a flowchart illustrating an embodiment of a multi-dimensionaldata processing process.

FIG. 6 is a flowchart illustrating another embodiment of amulti-dimensional data processing process.

FIG. 7 is a flowchart illustrating an embodiment of a tensor de-noisingprocess.

FIGS. 8A-8D are diagrams illustrating an example tensor and theintermediate results as it is de-noised.

FIG. 8E illustrates the equivalent tensor of the reduced SVD for theprimary tensor.

FIG. 8F shows the result of the IFFT for the primary tensor.

FIG. 8G illustrates the equivalent tensor of the reduced SVD for thesecondary tensor.

FIG. 8H shows the result of the IFFT for the secondary tensor.

FIGS. 8I-8K are diagrams illustrating an example of a set of actual datathat is de-noised using process 700.

FIG. 9 is a flowchart illustrating an embodiment of a clusteringprocess.

FIG. 10 is a flowchart illustrating an embodiment of a process usingFourier Transform to obtain a tensor convolution result.

FIGS. 11A-11D are diagrams illustrating an example tensor and theintermediate results as it is clustered.

FIG. 12A is a diagram illustrating one set of data collected for a groupof consumers given a specific set of spending volume, interest rate, andCCI values.

FIG. 12B is a diagram illustrating another set of data showing thepurchasing behaviors of the group of consumers with an additionaldimension, spending volume (which measures the amount of spending).

FIG. 12C is a diagram illustrating an example of raw data for thepurchasing behaviors of the group of consumers along multipledimensions.

FIG. 12D is a diagram illustrating the example raw data as shown in avisualization tool.

FIG. 12E is a diagram illustrating an example of a 5-dimensional tensorconstructed based on the raw data.

FIGS. 12F, 12G, and 12H are diagrams illustrating the clustered resultcorresponding to tensors 1202, 1204, and 1206, respectively, as shown bya visualization tool.

FIG. 13 is a diagram illustrating an example visual display of a slice(a result matrix) of the processed result tensor based on input tensor400 of FIG. 4.

FIG. 14A is a diagram illustrating an example input tensor constructedbased on raw data.

FIG. 14B is a diagram illustrating output tensor 1420.

FIG. 14C is a diagram illustrating a selected set of slices.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits, and/or processing coresconfigured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

Data processing is disclosed. In some embodiments, the data processingincludes accessing a matrix (M) representing a graph; performing anoperation on the matrix to generate a result matrix, the operationincluding a multiplication function on the matrix; and identifying oneor more clusters among the plurality of nodes based at least in part onthe result matrix, including detecting one or more vertices among theplurality of nodes using the result matrix. In some embodiments, thedata processing includes accessing a tensor; performing FourierTransform on the tensor to obtain a plurality of harmonic matrices;performing singular value decompositions (SVDs) on the plurality ofharmonic matrices to obtain a plurality of corresponding SVD results;reducing the plurality of SVD results, including selecting one or moredominant components in the plurality of corresponding SVD results toobtain a plurality of reduced results; and performing Inverse FourierTransform on the plurality of reduced results to obtain a de-noisedtensor that expresses stronger and clearer linkages of nodes asrepresented in the matrices forming the de-noised tensor. In someembodiments, the data processing includes accessing a tensor comprisinga plurality of matrices across one or more dimensions, a matrix in theplurality of matrices representing nodes that potentially haveinterconnections; applying a tensor product function to the tensor toobtain a tensor graph that indicates changes in interconnections ofnodes across the one or more dimensions; and outputting at least aportion of the graph.

FIG. 1 is a functional diagram illustrating a programmed computer systemfor performing data clustering in accordance with some embodiments. Aswill be apparent, other computer system architectures and configurationscan be used to perform clustering. Computer system 100, which includesvarious subsystems as described below, includes at least onemicroprocessor subsystem (also referred to as a processor or a centralprocessing unit (CPU)) 102. For example, processor 102 can beimplemented by a single-chip processor or by multiple processors. Insome embodiments, processor 102 is a general purpose digital processorthat controls the operation of the computer system 100. Usinginstructions retrieved from memory 110, the processor 102 controls thereception and manipulation of input data, and the output and display ofdata on output devices (e.g., display 118). In some embodiments,processor 102 includes and/or is used to execute/perform the processesdescribed below with respect to FIGS. 2, 5, 6, 7, 9, and 10.

Processor 102 is coupled bi-directionally with memory 110, which caninclude a first primary storage, typically a random access memory (RAM),and a second primary storage area, typically a read-only memory (ROM).As is well known in the art, primary storage can be used as a generalstorage area and as scratch-pad memory, and can also be used to storeinput data and processed data. Primary storage can also storeprogramming instructions and data, in the form of data objects and textobjects, in addition to other data and instructions for processesoperating on processor 102. Also as is well known in the art, primarystorage typically includes basic operating instructions, program code,data, and objects used by the processor 102 to perform its functions(e.g., programmed instructions). For example, memory 110 can include anysuitable computer-readable storage media, described below, depending onwhether, for example, data access needs to be bi-directional oruni-directional. For example, processor 102 can also directly and veryrapidly retrieve and store frequently needed data in a cache memory (notshown).

A removable mass storage device 112 provides additional data storagecapacity for the computer system 100, and is coupled eitherbi-directionally (read/write) or uni-directionally (read only) toprocessor 102. For example, storage 112 can also includecomputer-readable media such as magnetic tape, flash memory, PC-CARDS,portable mass storage devices, holographic storage devices, and otherstorage devices. A fixed mass storage 120 can also, for example, provideadditional data storage capacity. The most common example of massstorage 120 is a hard disk drive. Mass storages 112, 120 generally storeadditional programming instructions, data, and the like that typicallyare not in active use by the processor 102. It will be appreciated thatthe information retained within mass storages 112 and 120 can beincorporated, if needed, in standard fashion as part of memory 110(e.g., RAM) as virtual memory.

In addition to providing processor 102 access to storage subsystems, bus114 can also be used to provide access to other subsystems and devices.As shown, these can include a display monitor 118, a network interface116, a keyboard 104, and a pointing device 106, as well as an auxiliaryinput/output device interface, a sound card, speakers, and othersubsystems as needed. For example, the pointing device 106 can be amouse, stylus, track ball, or tablet, and is useful for interacting witha graphical user interface.

The network interface 116 allows processor 102 to be coupled to anothercomputer, computer network, or telecommunications network using anetwork connection as shown. For example, through the network interface116, the processor 102 can receive information (e.g., data objects orprogram instructions) from another network or output information toanother network in the course of performing method/process steps.Information, often represented as a sequence of instructions to beexecuted on a processor, can be received from and outputted to anothernetwork. An interface card or similar device and appropriate softwareimplemented by (e.g., executed/performed on) processor 102 can be usedto connect the computer system 100 to an external network and transferdata according to standard protocols. For example, various processembodiments disclosed herein can be executed on processor 102, or can beperformed across a network such as the Internet, intranet networks, orlocal area networks, in conjunction with a remote processor that sharesa portion of the processing. Additional mass storage devices (not shown)can also be connected to processor 102 through network interface 116.

An auxiliary I/O device interface (not shown) can be used in conjunctionwith computer system 100. The auxiliary I/O device interface can includegeneral and customized interfaces that allow the processor 102 to sendand, more typically, receive data from other devices such asmicrophones, touch-sensitive displays, transducer card readers, tapereaders, voice or handwriting recognizers, biometrics readers, cameras,portable mass storage devices, and other computers.

In addition, various embodiments disclosed herein further relate tocomputer storage products with a computer readable medium that includesprogram code for performing various computer-implemented operations. Thecomputer-readable medium is any data storage device that can store datawhich can thereafter be read by a computer system. Examples ofcomputer-readable media include, but are not limited to, all the mediamentioned above: magnetic media such as hard disks, floppy disks, andmagnetic tape; optical media such as CD-ROM disks; magneto-optical mediasuch as optical disks; and specially configured hardware devices such asapplication-specific integrated circuits (ASICs), programmable logicdevices (PLDs), and ROM and RAM devices. Examples of program codeinclude both machine code, as produced, for example, by a compiler, orfiles containing higher level code (e.g., script) that can be executedusing an interpreter.

The computer system shown in FIG. 1 is but an example of a computersystem suitable for use with the various embodiments disclosed herein.Other computer systems suitable for such use can include additional orfewer subsystems. In addition, bus 114 is illustrative of anyinterconnection scheme serving to link the subsystems. Other computerarchitectures having different configurations of subsystems can also beutilized. For example, in various embodiments, a client-serverarchitecture and/or a cloud-based architecture comprising multiplecomputer systems, virtual machines, or the like, can be used to providethe functions described below.

FIG. 2 is a flowchart illustrating an embodiment of a clusteringprocess. Process 200 can be performed by a system such as 100.

At 202, a matrix representing a graph is accessed. In this example, agraph represents interconnected nodes to be clustered, and a matrixrepresenting the graph can be obtained by following certain rules.

FIG. 3A is a diagram illustrating an embodiment of a graph. Graph 300 isa directed graph that is generated based on the nodes to be clusteredand their associations. In this example, graph 300 includes nodesidentified using identification numbers (ID) 1, 2, 3, etc., and edgessuch as 302, 304, 306, etc. The nodes represent objects or entities(e.g., users, organizations, places, etc.), and the edges represent theassociations between nodes. Where there is an association between twonodes, an edge is formed. In this case, each edge is directional, andrepresents an association between a source node from which the edgeoriginates and a destination node at which the edge terminates. In thediagram shown, the arrow representing an edge points away from thesource node and towards the destination node. Another way of looking atthis is that the source node is influenced by the destination node.

The construction of the graph depends on the context of the data beingprocessed and can require some domain knowledge. For example, supposethat the nodes represent users of a social networking platform. Where afirst user follows a second user, the node corresponding to the firstuser is a source node and the node corresponding to the second user is adestination node. As another example, the nodes represent airports, andan edge is formed between the departure airport and the arrival airportof a flight. Many other constructions are possible. For purposes ofdiscussion, it is assumed that for two-dimensional clustering, thegraphs are pre-constructed and provided to the clustering process.

The matrix M representing the graph is an N×N matrix, where Ncorresponds to the number of nodes. An entry at the (x, y) location inthe matrix is assigned a value of 1 if there is an edge from source nodex to destination node y, and is assigned a value of 0 if there is noedge from source node x to destination node y. For example, in FIG. 3A,there is an edge originating from node 1 to node 3, thus the matrixentry at (1, 3) is set to 1; and there is no edge originating from node3 to node 1, thus the entry at (3, 1) is set to 0. Further, all entrieson the diagonal of the matrix are set to 1 because each node is deemedto be self-connected (that is, having an edge originating from andending at itself), and therefore are set to 1.

Accordingly, the matrices M_(A), M_(B), M_(C), and M_(D) representingexample graphs 300, 320, 340, and 360 of FIGS. 3A-3D, respectively, are:

${M_{A} = \begin{bmatrix}1 & 1 & 1 \\0 & 1 & 1 \\0 & 0 & 1\end{bmatrix}};$ ${M_{B} = \begin{bmatrix}1 & 1 & 1 & 1 \\0 & 1 & 1 & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 1 & 1\end{bmatrix}};$ ${M_{C} = \begin{bmatrix}1 & 0 & 0 & 0 & 1 \\1 & 1 & 0 & 0 & 1 \\0 & 1 & 1 & 0 & 1 \\0 & 0 & 0 & 1 & 1 \\0 & 0 & 0 & 0 & 1\end{bmatrix}};{and}$ $M_{D} = {\begin{bmatrix}1 & 0 & 0 & 0 & 1 \\1 & 1 & 0 & 0 & 0 \\0 & 1 & 1 & 0 & 1 \\0 & 0 & 0 & 1 & 1 \\0 & 0 & 0 & 0 & 1\end{bmatrix}.}$

As described herein, two edges originating from two separate sourcenodes and terminating at the same destination node, and one or moreedges connecting the source nodes (referred to as the base) form avertex of a triangle at the destination node. A node at which a largenumber of vertices are formed is deemed to be more influential relativeto other nodes having fewer vertices. For example, a user on a socialnetwork who has a large number of followers or a seller on an e-commerceplatform who has a lot of customers would be deemed influential, andthis information is useful for clustering. In the example shown, thegraph also has self-connectivity. In other words, the graph can includeself-connected nodes (i.e., each node is connected to itself). Forexample, in a graph representing users of a social networking platform,each user is represented as a node in the graph that is self-connected.This is because each node is deemed to be influential to itself.

Returning to FIG. 2, at 204, an operation is performed on the matrix. Inthis example, the operation includes one or more multiplicationfunctions on the matrix.

In one example, the operation (OP) corresponds to squaring the matrix:OP=M ²  (1)

Accordingly, with example graphs 300-360, the results of the squaringoperations are:

${{OP}_{A} = {M_{A}^{2} = \begin{bmatrix}1 & 2 & 3 \\0 & 1 & 2 \\0 & 0 & 1\end{bmatrix}}};$ ${{OP}_{B} = {M_{B}^{2} = \begin{bmatrix}1 & 2 & 4 & 2 \\0 & 1 & 2 & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 2 & 1\end{bmatrix}}};$ ${{OP}_{C} = {M_{C}^{2} = \begin{bmatrix}1 & 0 & 0 & 0 & 2 \\2 & 1 & 0 & 0 & 3 \\1 & 2 & 1 & 0 & 3 \\0 & 0 & 0 & 1 & 2 \\0 & 0 & 0 & 0 & 1\end{bmatrix}}};{and}$ ${OP}_{D} = {M_{D}^{2} = {\begin{bmatrix}1 & 0 & 0 & 0 & 2 \\2 & 1 & 0 & 0 & 1 \\1 & 2 & 1 & 0 & 2 \\0 & 0 & 0 & 1 & 2 \\0 & 0 & 0 & 0 & 1\end{bmatrix}.}}$

At 206, clusters are identified based on the result matrix of theoperation. In this example, one or more vertices are detected based onthe result matrix, and clusters are formed in connection with thevertices. Specifically, an entry corresponding to a value v that atleast meets a threshold is deemed to correspond to a vertex of atriangle in the graph. The column index of the identified entrycorresponds to the identification number of the vertex node (alsoreferred to as an influential node). The clustering results can be sentto a display, to another analytical application such as a predictivetool to be further processed and/or displayed, or the like.

For the examples of FIGS. 3A-3D, the threshold is set to 3. Other valuescan be determined empirically for different sized matrices or matriceswith different weights.

In the result of M_(A) ², entry (1, 3) of the result matrix meets thethreshold; therefore, node 3 is deemed to be a vertex node. Further, thenumber of vertex/vertices at this node is determined to be (value−2). Inother words, (value −2) indicates the number of triangles for which thisnode acts as a vertex/vertices. Thus, at node 3, there is (3−2)=1vertex. In the original matrix, non-zero entries located on the samecolumn as the vertex entry correspond to nodes connected to the vertex.These non-zero entries can be identified by inspection. These connectednodes form a cluster. In this case, the non-zero entries in M_(A) are(1, 3), (2, 3), and (3, 3). Since (3, 3) indicates a self-connection,nodes 1 and 2 are deemed to be connected to (and in the same cluster as)node 3.

In the result of M_(B) ², entry (1, 3) meets the threshold; therefore,node 3 is deemed to be a vertex node, and nodes 1, 2, and 4 are deemedto be connected to (and in the same cluster as) node 3. Since entry (1,3) has a value of 4 and 4−2=2, it indicates that there are two verticesat node 3 (in other words, node 3 acts as vertices for two triangles).

In the result of M_(C) ², entries (2, 5) and (3, 5) meet the threshold;therefore, node 5 is deemed to be a vertex node. By inspecting M_(C),nodes 1, 2, and 3 are deemed to be connected to (and in the same clusteras) node 5, and since (3−2=1), node 5 acts as a vertex for a trianglewith respect to node 2, and acts as a vertex for another triangle withrespect to node 3. Note that while node 4 connects to node 5, becausenode 4 does not form a triangle with node 5 and entry (4, 5) does notmeet the threshold, the connection is deemed to be a weak one and node 4is not included in the same cluster as node 5.

In the result of M_(D) ², no entry meets the threshold. This is becausethere is no first order triangle (that is, a triangle with a basecomprising a single edge) in graph 360. A different operation can beperformed to identify vertices formed by second order triangles (thatis, a triangle with a base comprising two connected edges). In thiscase,OP=M ³  (2)

Accordingly,

${OP}_{D} = {M_{D}^{3} = \begin{bmatrix}1 & 0 & 0 & 0 & 3 \\3 & 1 & 0 & 0 & 3 \\3 & 3 & 1 & 0 & 4 \\0 & 0 & 0 & 1 & 3 \\0 & 0 & 0 & 0 & 1\end{bmatrix}}$

In this example, a threshold of 4 is used to identify the vertex. Entry(3, 5) meets this threshold, indicating that node 5 is a vertex for atriangle. In this example, according to the graph, node 5 has 2 incidentedges, one originating from node 1 and another one originating from node3, forming two sides of a triangle. The base side of the triangleincludes 2 segments (from node 3 to node 2 and from node 2 to node 1).

Other operations can be used to identify the vertices. In anotherexample, the operation is:OP=M·(M−I),  (3)

where I is the identity matrix. In this case, the threshold is 2. Thus,at 206, the same node determinations can be made based on the resultsusing function (1). Accordingly,

${OP}_{A} = {{M_{A} \cdot \left( {M_{A} - I} \right)} = \begin{bmatrix}0 & 1 & 2 \\0 & 0 & 1 \\0 & 0 & 0\end{bmatrix}}$${{OP}_{B} = {{M_{B} \cdot \left( {M_{B} - I} \right)} = \begin{bmatrix}0 & 1 & 3 & 1 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 0 \\0 & 0 & 1 & 0\end{bmatrix}}};$${{OP}_{C} = {{M_{C} \cdot \left( {M_{C} - I} \right)} = \begin{bmatrix}0 & 0 & 0 & 0 & 1 \\1 & 0 & 0 & 0 & 2 \\1 & 1 & 0 & 0 & 2 \\0 & 0 & 0 & 0 & 1 \\0 & 0 & 0 & 0 & 0\end{bmatrix}}};{and}$${OP}_{D} = {{M_{D} \cdot \left( {M_{D} - I} \right)} = {\begin{bmatrix}0 & 0 & 0 & 0 & 1 \\1 & 0 & 0 & 0 & 1 \\1 & 1 & 0 & 0 & 1 \\0 & 0 & 0 & 0 & 1 \\0 & 0 & 0 & 0 & 0\end{bmatrix}.}}$

The results obtained using formulas (1)-(3) are consistent with theresults obtained using equation (1). In other words, the same verticesare determined.

Other formulas can be used to identify vertices in other embodiments.For example, the following general formula can be used to identify j-thorder vertices and clusters:OP=M·(M−I)^(j)  (4)

where j=1, 2, 3, etc. The j-th order vertices correspond to vertices atbase-j triangles (e.g., triangles whose bases are formed by j edgesegments), and are helpful for identifying nodes that are influential tothe j-th degree.

Although 0 or 1 are assigned to the entries in the matrices discussed inthe above examples, in some embodiments, an edge can be assigned aweighted value other than 0 or 1 to indicate the degree of associationbetween the source node and the destination node. For example, a weightof 0.5 indicates a lesser degree of association than a weight of 2.6.Process 200 can still be used to identify the influential nodes and thethreshold value used to detect the vertices can be an empiricallydetermined value. In some embodiments, instead of using a threshold toidentify the vertices in 206, entries with the highest values (e.g., thetop five entries) are selected as corresponding to vertices.

In some embodiments, the graph includes one or more edges thatcorrespond to negative values, indicating that there are negativeassociations between certain source nodes and certain destination nodes,for example, when certain source users indicate dislike or disapprovalof certain target users. In such cases, process 200 can be used toconduct friend/foe analysis to identify unfriendly clusters and/orfriendly clusters. As used herein, a friendly cluster includes nodesthat are friendly towards (or are positively influenced by) aninfluential node of the cluster, and an unfriendly cluster (alsoreferred to as a foe cluster) includes nodes that are unfriendly towards(or are negatively influenced by) an influential node of the cluster.

At 202, a matrix representing the graph is accessed. Here, the graphincludes one or more negative edges indicating negative associationsbetween certain nodes and therefore the matrix includes one or moreentries.

At 204, an operation that includes a multiplication operation on thematrix is performed. Operations such as (1) or (2) can be performed toobtain the main vertices and identify the main clusters, (3) can beperformed to obtain the secondary vertices and identify the secondaryclusters, (4) can be performed to obtain the tertiary vertices andidentify the tertiary clusters, etc.

At 206, the clusters are identified based on the result of 204. In thiscase, a negative threshold (e.g., −2 for an initial matrix that hasentry values of 1 or −1) is used to identify unfriendly clusters (alsoreferred to as the foe clusters). The value of the negative threshold isempirically determined and can vary for edges having weighted values.Specifically, in some embodiments, an entry of the result of 204 thatexceeds (e.g., is less than or equal to) the negative threshold value isdeemed to correspond to an influential node, and the nodes connecting tothe determined influential node form a foe cluster. In some embodiments,one or more most negative entries (e.g., the bottom five most negativeentries) in the matrix in the result are selected as the mostinfluential node.

Additional inferences can be made based on the identified foe clustersto further process the nodes. For example, suppose that severalunconnected nodes are all found to be in a foe cluster with respect to aparticular influential node, a further inference can be drawn that thesedisconnected nodes are deemed to be friendly to each other and thereforeare clustered into a friendly cluster together.

In some embodiments, two thresholds, one positive and one negative, areused to identify friendly clusters and unfriendly clusters,respectively. The values of the thresholds are empirically determinedand can vary for different embodiments (e.g., 3 and −2). An entry thatexceeds (e.g., is greater than or equal to) the positive threshold haspositive influence on the nodes connecting to it (in other words, thenodes form a friendly cluster). An entry that exceeds the negativethreshold has negative influence on the nodes connecting to it (in otherwords, these nodes form an unfriendly cluster).

Multi-Variate and Multi-Dimensional Clustering

In the above discussion, the clustering is performed on multiple nodes(or equivalently, variables or variates) organized in a 2-dimensionalmatrix. In many situations the nodes influence each other acrossmultiple dimensions. In one example, the nodes represent consumers andbrands of products they purchase. The dimensions can be temperature,unemployment rate, exchange rate, etc., all of which can influencepurchasing behaviors of the consumers. As another example, the nodesrepresent airports, and the dimensions can be delays, time of the day,days of the week, etc. Historical data can be recorded and matricesrepresenting nodes and their interconnections are formed across variousdimensions. How to construct the matrices often requires domainknowledge and depends on the data being analyzed. As will be describedin greater detail below, data recorded as multi-variate andmulti-dimensional matrices is input to a clustering process to beclustered. The clustering process removes noise and establishes moreclear linkages between nodes across multiple dimensions. In particular,the multi-variate and multi-dimensional clustering process determinesthe effects of one or more varying dimensions on the clusters. Forexample, how sensitive are people's purchasing behaviors to changes intemperature, unemployment range, exchange rate, etc., change; how doairports affect each other in terms of delays with changes in time ofthe day, day of the week, etc.

As used herein, a 1-dimensional matrix (an array) is referred to as avector; a 2-dimensional matrix is referred to simply as a matrix; a3-dimensional or higher matrix is referred to as a tensor. In a tensor,one or more axes represent the variables (also referred to as variates),and other axes represent the dimensions over which the variates admitmoments or rates of change. For purposes of illustration, 3-dimensionaltensors are discussed in the examples below. The techniques describedalso apply to tensors of four or more dimensions.

In the examples below, it is assumed that a tensor comprising2-dimensional matrices across one or more dimensions (e.g., across the3^(rd) or higher dimensions) has been constructed and is to beprocessed. FIG. 4 is a diagram illustrating a visualization of amulti-dimensional tensor example relating to airport delays. In thisexample, 2-dimensional matrices are formed according to the dimensionsof departure airports and arrival airports. A third dimensioncorresponds to the amount of delays, and a 3-dimensional tensor can beconstructed as stacking multiple 2-dimensional matrices across the thirddimension. A fourth dimension corresponds to arrival time of the day,and a 4-dimensional tensor can be constructed as multiple 3-dimensionaltensors stacked across the fourth dimension. The fifth dimensioncorresponds to day of the week, the sixth dimension corresponds toairline, and a 5-dimensional tensor can be constructed as multiple4-dimensional tensors stacked across the fourth dimension, and so on. Aswill be shown in greater detail below, the 3-dimensional tensors can beprocessed using efficient clustering techniques. Thus, tensors of fourdimensions or higher can be processed similarly by processing the3-dimensional tensors that form the higher dimensional tensors.

Generally, an N-dimensional tensor (N being an integer greater than 2)can be constructed by stacking a set of (N−1)-dimensional tensors alongthe N-th dimension, or equivalently, by stacking matrices along multipledimensions. Data can be collected over a period of time (e.g., severalmonths) and put into bins based on the corresponding dimensions (e.g.,delay of 0 hours, delay of 0.5 hours, delay of 1 hour, etc., arrivingbetween 9 am-noon, noon-3 pm, 3 pm-6 pm, etc., on Monday, Tuesday,Wednesday, etc.) to construct the tensor to be processed.

FIG. 5 is a flowchart illustrating an embodiment of a multi-dimensionaldata processing process. Process 500 can be performed by a system suchas 100.

At 502, a tensor is accessed. As discussed above, the tensor is amulti-dimensional matrix representing the raw data to be clustered. Theraw data can be collected and stored separately from process 500. Atthis point, the tensor is said to be in a “native domain.” Anidentifier, pointer, handle, reference, or the like to a storage ormemory location of the tensor can be used to access the tensor andobtain values of its entries.

In this process, the tensor is deemed to have noise. Thus, at 504, thetensor is de-noised to generate a de-noised tensor. As will be describedin greater detail below in connection with FIG. 7, the de-noisingincludes transforming the original tensor to the Fourier domain usingFourier transform, performing an N-dimensional spectral reduction (orequivalently, an SVD in the Fourier domain), then performing an inverseFourier transform to return to the native domain. The de-noised tensorhas the same dimensions as the original tensor accessed at 502.

At 506, the de-noised tensor is clustered. As will be described ingreater detail below in connection with FIG. 9, the clustering includesapplying a tensor product function to the de-noised tensor. The tensorproduct function can be a convolution, or a multiplication in theFourier domain converted back to the native domain. The result is ade-noised, clustered tensor having the same dimensions as the originaltensor.

FIG. 6 is a flowchart illustrating another embodiment of amulti-dimensional data processing process. Process 600 can be performedby a system such as 100.

At 602, the tensor is accessed. As discussed above, the tensor is amulti-dimensional matrix representing the raw data in the native domain.An identifier, pointer, handle, reference, or the like to a storage ormemory location of the tensor can be used to access the tensor andobtain values of its entries.

In this process, the interconnections of nodes represented by the tensorare to be preserved as much as possible. Thus, any cleanup (de-noising)of data is to occur after the clustering. At 604, the tensor isclustered to generate a clustered result. Details of the clustering aredescribed below in connection with FIG. 9. The clustering of the tensorincludes applying a tensor product to the tensor. In some embodiments,the tensor product operation includes convolution; in some embodiments,the tensor product operation includes transforming the tensor to theFourier domain, multiplying the transformed result with its transpose,then inverse transforming the multiplication result back into the nativedomain. The clustered result is a tensor having the same dimensions asthe original tensor accessed at 602.

At 606, the clustered result is de-noised. Details of the de-noising aredescribed below in connection with FIG. 7. The result is a clustered,de-noised tensor having the same dimensions as the original tensor.

While both processes 500 and 600 provide ways for clustering data andestablishing linkages across multiple dimensions, they performde-noising and clustering in different orders. When raw data isrelatively noisy and filtering out the noise would not significantlyimpact the type of linkages/clusters being sought, process 500 isselected. For example, suppose that the clusters to be establishedpertain to people's purchasing behavior towards certain consumer productbrands, and the raw data also includes various information deemedirrelevant towards this end (e.g., if a consumer makes a purchase of aparticular brand of product once every ten years, but makes purchases ofa different brand of the same type of product on a monthly basis, thefirst purchase may be deemed irrelevant in one clustering process).Process 500 is preferably used to de-noise the data first to filter outthe irrelevant information, and allow the clustering to be performedmore efficiently and to provide a cleaner result. As another example,suppose that the clusters to be established pertain to the prices of aset of stocks, and it is initially unclear how different factors in theraw data influence the stock prices. Process 600 is preferably used tocluster the data first so as to preserve the interconnections betweennodes as much as possible and provide a more accurate result. Further,in some embodiments, the raw data can be de-noised, clustered, thende-noised again.

FIG. 7 is a flowchart illustrating an embodiment of a tensor de-noisingprocess. FIGS. 8A-8D are diagrams illustrating an example tensor and theintermediate results as it is de-noised. Process 700 is explained inconnection with 8A-8D for purposes of example.

Process 700 can be used to implement, for example, 504 of process 500 or606 of process 600 (in which case the input tensor would be theclustered tensor resulting from 604). Depending on the context in which700 is invoked, the initial tensor to be processed by flow 700 cancorrespond to raw data (e.g., the original tensor accessed by 502 ofprocess 500) or clustered data (e.g., the clustered result generated by604 of process 600).

The tensor to be processed includes multiple matrices and forms aplurality of vectors across one or more dimensions. FIG. 8A illustratesa 3-dimensional tensor example. In this example, tensor 800 includes MN×N matrices such as A₁, A₂, . . . A_(M), etc. (M and N are integers)across a dimension along the Z-axis. To facilitate subsequentprocessing, entries at the same corresponding locations in the matricesare grouped across the dimension Z into vectors. For example, entries atlocation (1, 1) of the matrices are grouped to form a vectorV _(1,1)=(a _(1,1,1) ,a _(1,1,2) , . . . a _(1,1,M))

Entries at location (1, 2) of the matrices are grouped to form a vectorV _(1,2)(a _(1,2,1) ,a _(1,2,2) , . . . a _(1,2,M)).

Entries at location (i, j) of the matrices are grouped to form a vectorV _(i,j)=(a _(i,j,1) ,a _(i,j,2) , . . . a _(i,j,M)), and so on.

Subsequent processing of the vectors allows for faster computation andreduces memory requirement in comparison to traditional clusteringtechniques.

Returning to FIG. 7, at 704, Fourier Transform is applied to the tensorto obtain a plurality of harmonic matrices in the Fourier domain.Specifically, discrete Fourier transform operations such as Fast FourierTransform (FFT) are applied to the vectors in the tensor (e.g., V_(1,1),V_(1,2), . . . , etc.) to obtain a plurality of correspondingtransformed vectors in the Fourier domain. Examples of FFT techniquesinclude Cooley-Tukey FFT, Prime-Factor FFT, and Rader's FFT. Any otherappropriate discrete Fourier transform technique can be used.

Each transformed vector generated by the FFT will also have M entries.In other words:

FFT (V_(i,j))=V′_(i,j)=(a′_(i,j,1), a′_(i,j,2) . . . a′_(i,j,M)), wherei=1, 2, . . . N, j=1, 2, . . . , N, V′_(i,j) is the transformed vectorand a′_(i,j,1), a′_(i,j,2), . . . , a′_(i,j,M) are entries of thetransformed vector. The entries are also referred to as the signalcomponents or harmonics. In particular, a′_(i,j,1) is referred to as afundamental or base component, a′_(i,j,2) is referred to as the firstharmonic, a′_(i,j,k) is referred to as the (k−1)th harmonic, etc. Thetransformed vectors are arranged to form a new set of M N×N harmonicmatrices (A′₁, A′₂, . . . A′_(M)) where the harmonics of the same orderform corresponding harmonic matrices. Since the vectors can undergo FFTindependently, the FFT processing can be done in parallel (e.g., carriedout contemporaneously on multiple processors or processor cores), thusgreatly improving the processing speed.

As shown in FIG. 8B, the entries are indexed according to theirpositions in tensor 820. For example, a′_(i,j,k) is located in the i-throw and j-th column of the k-th harmonic matrix A′_(k). The set of baseentries a′_(1,1,1), . . . a′_(1,N,1) . . . , a′_(N,1,1), . . . ,a′_(N,N,1), etc. collectively forms the base or fundamental harmonicmatrix A′₁; the set of first harmonic entries a′_(1,1,2), . . .a′_(1,N,2) . . . , a′_(N,1,2), . . . a′_(N,N,2), etc. collectively formsthe first harmonic matrix A′₂, and so on.

To facilitate understanding, the tensor transformation is explained bymaking comparisons to traditional signal processing techniques. Intraditional signal processing, an original signal in the time domaincontains certain frequency components which are difficult to discern inthe time domain. Fourier Transform transforms the original signal intothe frequency domain, making the spectral components more distinct andoften easier to process from a frequency stand point. In process 700,prior to the FFT, the original matrices have certain spectral componentsin the native domain (akin to signals in the time domain havingfrequency components). In the native domain, the spectral components andtheir relationships are hard to discern. Transforming the originalmatrices into the Fourier domain makes the spectral components and theirrelationships clearer and easier to process. For instance, A′₁corresponds to the fundamental spectral component, A′₂ corresponds tothe secondary spectral component, A′_(k) corresponds to the k-thspectral component, etc.

At 706, Singular Value Decompositions (SVD) is performed on the harmonicmatrices to obtain a plurality of corresponding SVD results. SVDcomputational techniques such as reducing the matrix to a bidiagonalmatrix then computing the eigenvalues or any other appropriate SVDcomputational techniques known to those skilled in the art can beapplied. According to the standard formula for SVD, a harmonic matrixA′_(k) is decomposed into the following form:

${{SVD}\left( A_{k}^{\prime} \right)} = {{{\left\lbrack {u_{1,k},u_{2,k},{\ldots\mspace{11mu} u_{N,k}}} \right\rbrack\begin{bmatrix}\sigma_{1,k} & \; & \; & \; \\\; & \sigma_{2,k} & 0 & \; \\\; & 0 & \ddots & \; \\\; & \; & \; & \sigma_{N,k}\end{bmatrix}}\begin{bmatrix}v_{1,k}^{T} \\v_{2,k}^{T} \\\ldots \\v_{N,k}^{T}\end{bmatrix}} = {{u_{1,k}\sigma_{1,k}v_{1,k}^{T}} + {u_{2,k}\sigma_{2,k}v_{2,k}^{T}} + \ldots\mspace{11mu} + {u_{N,k}\sigma_{N,k}v_{N,k}^{T}}}}$

where σ_(i,k) is the i-th singular value, u_(i,k) is the i-th leftsingular vector, and v_(i,k) is the i-th right singular vector. Theterms are preferably sorted and ordered according to the singularvalues, such that σ_(i,k) is the greatest, σ_(2,k) is the secondgreatest, etc. FIG. 8C illustrates the SVD results corresponding tomatrices A′₁, A′₂, . . . , A′_(k), . . . A′_(N) of FIG. 8B.

At 708, the SVD results are reduced. Specifically, one or more dominantcomponents in the SVD results are selected to obtain one or more reducedresults. Specifically, terms with the P highest values are selected andother terms are set to 0. The value of P is specified by the useraccording to the requirements of the application. For example, assumingthe terms in the SVD are sorted, if P=2, the reduced expressions are:for SVD(A′ ₁):u _(1,1)σ_(1,1) v _(1,1) ^(T) v+u _(2,1) v _(2,1) ^(T)for SVD(A′ ₂):u _(1,2)σ_(1,2) v _(1,2) ^(T) +u _(2,2)σ_(2,2) v _(2,2)^(T). . .for SVD(A′ _(k)):u _(1,k)σ_(1,k) v _(1,k) ^(T) +u _(2,k)σ_(2,k) v _(2,k)^(T). . .for SVD(A′ _(M)):u _(1,M)σ_(1,M) v _(1,M) ^(T) +u _(2,M)σ_(2,M) v _(2,M)^(T).

FIG. 8D illustrates the reduced SVD of FIG. 8C. In this case, the firstterms combine to form a primary tensor B:

$B = \begin{bmatrix}{u_{1,1}\sigma_{1,1}v_{1,1}^{T}} \\{u_{1,2}\sigma_{1,2}v_{1,2}^{T}} \\\ldots \\{u_{1,k}\sigma_{1,k}v_{1,k}^{T}} \\\ldots \\{u_{1,M}\sigma_{1,M}v_{1,M}^{T}}\end{bmatrix}$

Also, the second terms combine to form a secondary tensor C:

$C = \begin{bmatrix}{u_{2,1}\sigma_{2,1}v_{2,1}^{T}} \\{u_{2,2}\sigma_{2,2}v_{2,2}^{T}} \\\ldots \\{u_{2,k}\sigma_{2,k}v_{2,k}^{T}} \\\ldots \\{u_{2,M}\sigma_{2,M}v_{2,M}^{T}}\end{bmatrix}$

FIG. 8E illustrates the equivalent tensor of the reduced SVD for theprimary tensor. As shown, tensor 880 comprises M N×N matrices.

FIG. 8G illustrates the equivalent tensor of the reduced SVD for thesecondary tensor. As shown, tensor 850 also comprises M N×N matrices.

Returning to FIG. 7, at 710, inverse Fourier transforms (e.g., discreteinverse Fourier transform such as Inverse Fast Fourier Transforms(IFFT)) are performed on the reduced results to generate a de-noisedtensor. Similar to the FFT process, the entries at the samecorresponding locations in the matrices in each tensor are grouped toform a vector on which an IFFT is performed.

Referring to FIG. 8E, the IFFT is applied to vectors such as W_(1,1),W_(1,2), . . . , W_(i,j), . . . , etc. Techniques such as Cooley-TukeyIFFT, Prime-Factor IFFT, Rader's IFFT, or any other appropriate IFFTtechnique can be used. Entries of the IFFT result vectors (e.g., entriesof vectors W′_(1,1), W′_(1,2), etc.) occupy the corresponding locationsin the tensor as the corresponding entries of the input vector to theIFFT function. FIG. 8F shows the result of the IFFT for the primarytensor. Similarly, FIG. 8H shows the result of the IFFT for thesecondary tensor. Tensors 890 and 895 can be output to a graphing oranalysis tool, to a clustering process such as 900 described below, orthe like.

FIGS. 8I-8K are diagrams illustrating an example of a set of actual datathat is de-noised using process 700. FIG. 8I illustrates the original,raw data to be processed. FIG. 8J illustrates the de-noised primarytensor. FIG. 8K illustrates the de-noised secondary tensor. As can beseen from the figures, noise from the raw data is successfully removedand the correlations between nodes are more clearly shown.

FIG. 9 is a flowchart illustrating an embodiment of a clusteringprocess. Process 900 can be used to implement, for example, 506 ofprocess 500 to cluster the de-noised tensor, or 604 of process 600 tocluster a tensor before it is de-noised.

In this example, the tensor to be clustered, represented as

, includes a plurality of matrices across one or more dimensions. Inthis example,

includes M N×N matrices A₁, A₂, . . . A_(M), which are 2-dimensionalmatrices laid out across a third dimension Z. The matrices representnodes that potentially have interconnections. Changes in a node alongdimension Z can affect other nodes; however, such interconnections canbe difficult to discern in the native domain.

At 902, a tensor product function is applied to the tensor

to obtain a tensor graph

that indicates the strengths of interconnections of nodes across the oneor more directions.

In this example, the tensor product function includes convolving thetensor with its transpose:

=

*

^(T),

where * represents a convolution function.

At 904, clusters in the matrices in the result tensors are optionallyidentified. The result of the tensor product includes certain peakvalues. These peak values indicate that the nodes corresponding to thepeak matrix entries are the influencers relative to other nodes. In thiscase, vertices in matrices in the result tensor (e.g., B′_(j) of tensor890 or C′_(j) of tensor 895) are identified using the techniquesdescribed above in connection with FIG. 2 (e.g., comparing with athreshold value, selecting the peak values, etc.), and nodes connectedto the identified vertices are grouped together. As discussed above, thevertices identified can be first order vertices, second order vertices,or generally j-th order vertices.

In some cases, such as when process 900 is invoked before the de-noisingprocess, 904 can be optionally postponed after the de-noising iscompleted. At 906, at least a portion of the tensor graph is output tobe displayed and/or further processed by another analytic tool orengine. If clusters are identified, cluster information can also beoptionally output. In some embodiments, the tensor graph is output to ade-noising stage, e.g., a process such as 700.

In some embodiments, to compute the convolution, a series ofsum-of-products are computed as follows:

=[G ₁ ,G ₂ , . . . ,G _(M)] whereG ₁ =A ₁ A ₁ ^(T) +A ₂ A ₂ ^(T) +A ₃ A ₃ ^(T) + . . . +A _(M−1) A _(M−1)^(T) +A _(M) A _(M) ^(T)G ₂ =A ₁ A ₂ ^(T) +A ₂ A ₃ ^(T) +A ₃ A ₄ ^(T) + . . . +A _(M−1) A _(M)^(T) +A _(M) A ₁ ^(T). . .G _(k) =A ₁ A _(k) ^(T) +A ₂ A _(k+1) ^(T) +A ₃ A _(k+2) ^(T) . . . +A_(M−1) A _(k+M−2) ^(T) +A _(M) A _(k+M−1) ^(T). . .G _(M) =A ₁ A _(M) ^(T) +A ₂ A ₁ ^(T) +A ₃ A ₂ ^(T) . . . +A _(M−1) A_(M−2) ^(T) +A _(M) A _(M−)1

where when the value of the subscript exceeds M, a modulus of the valueis computed such that the subscript has a value less than or equal to M(e.g., k+M−2 corresponds to a subscript of k−2).

The above convolution computation requires many shift and multiplicationoperations and can be computationally intensive. In some embodiments, tomore efficiently compute the convolution, the Fourier Transform of thetensor and the Fourier Transform of the tensor's transpose aremultiplied such that the computations mostly require multiplications ofvectors, and then an Inverse Fourier Transform is applied to themultiplication result to bring the result to the native domain. As willbe shown below, the Fourier Transforms and Inverse Fourier Transformsinvolve fewer shift and multiplication operations, and can be performedin parallel on separate processors/processor cores, thus resulting insignificant performance improvement over the sum-of-product technique.

FIG. 10 is a flowchart illustrating an embodiment of a process usingFourier Transform to obtain a tensor convolution result. Process 1000can be used to implement 902 of process 900 and produces an equivalentresult as the convolution function described above. FIGS. 11A-11D arediagrams illustrating an example tensor and the intermediate results asit is clustered. Process 1000 is explained in connection with FIGS.11A-11D for purposes of example.

At 1002, Fourier transform (e.g., a discrete Fourier transform such asan FFT) is performed on the input tensor. For example, in process 500,the input tensor corresponds to the de-noised tensor that is to beclustered; in process 600, the input tensor corresponds to the originaltensor to be clustered.

Details regarding how to perform FFT on a tensor are discussed above inconnection with 704 of FIG. 7. Referring to FIGS. 11A-11B for anexample, tensor 1100 in FIG. 11A corresponds to the input tensor whichcomprises matrices A₁, A₂, . . . A_(M) along the Z dimension, and tensor1120 in FIG. 11B corresponds to the tensor that has undergone FFT, whichcomprises harmonic matrices A′₁, A′₂, . . . , A′_(M).

At 1004, in the Fourier domain, the FFT result is multiplied with itstranspose conjugate to generate a product. In this case, themultiplication includes multiplying each matrix in the tensor with itscorresponding transpose conjugate (also referred to as a Hermitiantranspose). The product includes multiple product matrices expressed as:Q ₁ =A′ ₁·(A′ ₁)^(H)Q ₂ =A′ ₂·(A′ ₂)^(H). . .Q _(M) =A′ _(M)·(A′ _(M))^(H)

FIG. 11C is a diagram illustrating an example of the product matricesgenerated in 1004 of process 1000. Each Q_(k) corresponds to an N×Nmatrix.

At 1006, an Inverse Fourier Transform (e.g., a discrete inverse Fouriertransform such as an IFFT) is applied to the product to generate theclustered result. Specifically, IFFTs are applied to vectors in theproduct matrices to generate inversely transformed vectors which arearranged to form the result matrices (also referred to as the graphmatrices) in the native domain. As shown in the examples of FIGS.11C-11D, the entries at the same corresponding locations in the matricesin the tensor are grouped to form a plurality of vectors across the Zdimension (e.g., q_(1,1,1), q_(1,1,2) . . . q_(1,1,M) form vectorW_(1,1), q_(1,2,1), q_(1,2,2) . . . , q_(1,2,M) form vector W_(1,2),etc.). An IFFT is performed on the vector to generate a result vector inthe native domain (e.g., W′_(1,1), W′_(1,2), etc.). Entries of the IFFTresult vectors occupy the same corresponding locations in result tensor1180 as the entries of the input vectors in product tensor 1170.

The first result matrix in the native domain (e.g., Q′₁ of tensor 1180of FIG. 11D) indicates how the nodes relate to each other at the baselevel. Peaks in the values indicate strong interconnections. Forexample, if q′_(3,2,1) has a value that is higher compared with othervalues, then it can be inferred that nodes 3 and 2 have a high degree ofinterconnectivity at the base level. The subsequent result matrices(e.g., Q′₂, Q′₃, etc.) correspond to harmonics indicating how therelationships change along the changing dimension.

In some embodiments, the densities of the result matrices are optionallycomputed and compared to provide information about how significant aparticular matrix is in terms of its impact on the interconnections ofnodes along a particular matrix dimension. The density of the matrix canbe computed based at least in part on the energy of the matrices. Oneexample expression of the density for the k-th matrix is:Density_(k) =F _(k) /F _(total)  (4)

where F_(k) is the Frobenius norm for the k-th matrix and is computedas:

$\begin{matrix}{F_{k} = \sqrt{\sum\limits_{i = 1}^{N}\;{\sum\limits_{j = 1}^{N}\;{a_{i,j}}^{2}}}} & (5)\end{matrix}$

where a_(i,j) is the entry (i, j) of the k-th matrix;

F_(total) is the sum of the Frobenius norms of the M result matrices inthe result tensor.

If the density of the k-th matrix is greater than the other resultmatrices, the k-th level is deemed to have the most significant impacton the behaviors (e.g., interconnections) of the nodes in dimension Z.On the other hand, if the density of the k-th matrix is less than theother result matrices, then this level has little impact on thebehaviors of the nodes in dimension Z.

Although the above examples illustrate multi-variate, multi-dimensionalmatrix processing for 3-dimensional tensors, tensors of greaterdimensions can be processed similarly. For example, multi-dimensionaltensor 400 of FIG. 4 can be processed by performing processes 500 or 600on individual 3-dimensional tensors such as 402. The 3-dimensionaltensors can be processed in parallel to achieve high processing speed,or serially at lower processing speed but requiring smaller amounts ofhardware resources.

Some examples are now described to illustrate how to use the processesdescribed above to cluster multi-variate, multi-dimensional tensors.

In one example, purchasing behaviors of consumers are clustered todetermine how purchasing behaviors of consumers and brands change acrossmultiple dimensions such as spending volume, interest rate, defaultprofile, consumer confidence index (CCI), etc. FIG. 12A is a diagramillustrating one set of data collected for a group of consumers given aspecific set of spending volume, interest rate, and CCI values. Nodessuch as 1202 represent consumers, and nodes such as 1204 representvarious categories of consumer purchases, brands, etc. When a consumermakes a purchase relating to a particular category or brand of product,an edge is formed between the consumer node and the purchasedcategory/brand to indicate that the purchased category/brand exertsinfluence on the consumer node. A matrix can be formed based on thegraph.

FIG. 12B is a diagram illustrating another set of data showing thepurchasing behaviors of the group of consumers with an additionaldimension, spending volume (which measures the amount of spending). Asshown, the data set is collected for varying volumes while keeping thevalues of other dimensions (e.g., interest rate, default profile, CCI)constant. A matrix can be constructed for a corresponding discretespending volume. Additional sets of data can be collected with differentcombinations of values in other dimensions.

To account for other factors that influence purchasing behavior, datasets corresponding to different values in other dimensions such asinterest rates, consumer confidence index, default profile (whichmeasures the likelihood of default and can be a credit score or thelike), etc. can also be collected. FIG. 12C is a diagram illustrating anexample of raw data for the purchasing behaviors of the group ofconsumers along multiple dimensions. In this example, a large number ofconsumer purchasing activities are collected over a period of time, andthe data points are placed into bins along each additional dimension(e.g., spending volume of $100-$200 per month, $201-$300 per month,etc.; interest rate of 3%-3.25%, 3.26%-3.5%, etc.; CCI of 90-95, 96-100,etc., default profile of 5%-10%, 11%-15%, etc.).

FIG. 12D is a diagram illustrating the example raw data as shown in avisualization tool. The diagram shows how nodes are connected to certaincategories/brands without any clustering. The data is highly noisy, andthere is no apparent pattern of connections. Note that the axes in thediagram are produced by the visualization tool and do not necessarilycorrespond to any of the dimensions of the data as discussed above.

FIG. 12E is a diagram illustrating an example of a 5-dimensional tensorconstructed based on the raw data. Note that the dimensions of thetensor can be constructed based on the raw data but do not necessarilyneed to match the dimensions of the raw data.

In this case, the data is relatively noisy, thus process 500 is appliedto the 5-dimensional tensor to de-noise the data then cluster along eachdimension. Specifically, 3-dimensional tensors such as 1202, 1204, 1206,etc. are individually processed by process 500. The tensors can beprocessed in parallel.

FIGS. 12F, 12G, and 12H are diagrams illustrating the clustered resultcorresponding to tensors 1202, 1204, and 1206, respectively, as shown bya visualization tool. Boxes such as 1250 are the influential nodes(e.g., brands of products) around which user nodes are clustered. Again,the axes in the diagram are produced by the visualization tool and donot necessarily correspond to any of the dimensions of the datadiscussed above.

In another example, airport delay data is collected and clustered. Aspreviously explained, FIG. 4 shows an example set of multi-dimensionalairport delay data organized into a 6-dimensional tensor. To determinewhich departure cities have the greatest impact on arrival cities,process 600 is applied to the 6-dimensional tensor. Specifically,process 600 is applied to each 3-dimensional tensor in FIG. 4. Theresults are 3-dimensional tensors each comprising “slices” of2-dimensional matrices, and clustering can be done for each slice usingthe technique discussed above in connection with FIG. 2. FIG. 13 is adiagram illustrating an example visual display of a slice (a resultmatrix) of the processed result tensor based on input tensor 400 of FIG.4. The matrix being displayed shows how airports are correlated for aparticular delay bin. In this case, the clustering shows how certainairports are more influential in causing delays at other airports. Forexample, O'Hare airport, which has the strongest impact on delays atother airports, is shown to be the vertex for a cluster.

In a third example, stocks and their prices over the dimensions of timeand Dollar-Euro exchange rate are collected to determine the movementsof stocks.

FIG. 14A is a diagram illustrating an example input tensor constructedbased on raw data. The tensor indicates the changes in the prices ofvarious stocks as time passes and as the exchange rate changes. Somestocks are more affected by time than others (e.g., the prices can bemore volatile as the end of the quarter or the end of the yearapproaches), and some stocks are more affected by the exchange rate.

Tensor 1400 is input into process 500 of FIG. 5 to produce an outputtensor 1420. FIG. 14B is a diagram illustrating output tensor 1420. Toillustrate the effects of Dollar-Euro exchange rate on the movements ofstocks relative to each other, slices of the output tensor are selectedwhere the exchange rate varies while other factors stay constant. FIG.14C is a diagram illustrating a selected set of slices. Slices 1422-1430are selected from tensor 1420 and their corresponding plots aredisplayed to show the effects of Dollar-Euro exchange rate has on thestocks. As shown, the exchange rate has the greatest effect on the stockthat corresponds to peak 1450 of plot 1422, since this entry changes themost as the exchange rate changes. In contrast, the stock thatcorresponds to peak 1452 of plot 1420 is considered to be relativelyunaffected by the exchange rate. The analysis results can also be outputto a predictive engine which constructs a model for the stocks and makespredictions about their movements. The implementation of the predictiveengine is outside the scope of this application.

Clustering data has been disclosed. The technique described provides ananalytical solution to data clustering, is computationally moreefficient than existing techniques, and allows for multi-variate,multi-dimensional data to be easily clustered.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

What is claimed is:
 1. A method, comprising: accessing input dataincluding measured and/or recorded data associated with a plurality ofobjects or entities, the measured and/or recorded data being used toconstruct a tensor in a native domain, wherein the tensor: is a tensorof three or more dimensions, represents interconnections of nodes acrossone or more dimensions, comprises a plurality of matrices thatcorresponds to collected data, and forms a plurality of vectors acrossat least one of the one or more dimensions; applying a plurality ofFourier Transforms on the tensor to obtain a plurality of harmonicmatrices in a Fourier domain, wherein the tensor includes a plurality of2-dimensional matrices; and the applying of the plurality of FourierTransforms includes using a plurality of processors to performoperations in parallel on data in the plurality of 2-dimensionalmatrices; performing singular value decompositions (SVDs) on theplurality of harmonic matrices to obtain a plurality of correspondingSVD results; reducing the plurality of corresponding SVD results,including selecting one or more dominant components in the plurality ofcorresponding SVD results to obtain one or more reduced SVD results;performing a plurality of Inverse Fourier Transforms on the one or morereduced SVD results to obtain a de-noised tensor that representsde-noised data; and outputting the de-noised tensor to be displayed orfurther processed.
 2. The method of claim 1, wherein the applying of theplurality of Fourier Transforms to obtain the plurality of harmonicmatrices includes applying a plurality of Fast Fourier Transforms (FFTs)to the plurality of vectors to obtain a plurality of transformed vectorsand establishing the plurality of harmonic matrices.
 3. The method ofclaim 1, wherein the de-noised tensor is further clustered.
 4. Themethod of claim 1, wherein the de-noised tensor is further clustered,including by: applying a tensor product function to the de-noised tensorto obtain a tensor graph.
 5. The method of claim 1, wherein thede-noised tensor is further clustered, including by: applying a tensorproduct function to the de-noised tensor to obtain a tensor graph; andidentifying one or more clusters in the tensor graph.
 6. The method ofclaim 1, wherein the de-noised tensor is further clustered, includingby: applying a tensor product function to the de-noised tensor to obtaina tensor graph; and wherein applying the tensor product function to thede-noised tensor includes convolving the tensor with its transpose. 7.The method of claim 1, wherein the de-noised tensor is furtherclustered, including by: performing a plurality of Fourier Transforms onthe de-noised tensor to obtain a Fourier Transform result tensorcomprising a plurality of Fourier Transform result matrices; multiplyingthe Fourier Transform result tensor with its transpose conjugate toobtain a product; and applying a plurality of Inverse Fourier Transformsto the product to generate a tensor graph.
 8. The method of claim 1,wherein the de-noised tensor is further clustered, including by:performing a plurality of Fourier Transforms on the de-noised tensor toobtain a Fourier Transform result tensor comprising a plurality ofFourier Transform result matrices, including by performing Fast FourierTransforms on a plurality of vectors in the de-noised tensor; andmultiplying the Fourier Transform result tensor with its transposeconjugate to obtain a product, including by multiplying a pluralitymatrices in the Fourier Transform result tensor with a correspondingplurality of transpose conjugates to obtain a plurality of productmatrices.
 9. The method of claim 1, further comprising: clustering thede-noised tensor; and comparing densities of matrices in the de-noisedtensor.
 10. The method of claim 1, wherein the tensor is of fourdimensions or higher.
 11. The method of claim 1, wherein the accessingof the input data includes accessing the tensor using an identifier, apointer, a handle, or a reference to a storage or memory location of thetensor.
 12. The method of claim 1, wherein at least some of theplurality of Fourier Transforms, at least some of the plurality ofInverse Fourier Transforms, or both, are performed in parallel on aplurality of processors.
 13. A system, comprising: a plurality ofprocessors; and one or more memories coupled with the plurality ofprocessors, wherein the one or more memories are configured to providethe plurality of processors with instructions which when executed causethe plurality of processors to: access input data including measuredand/or recorded data associated with a plurality of objects or entities,the measured and/or recorded data being used to construct a tensor in anative domain, wherein the tensor: is a tensor of three or moredimensions, represents interconnections of nodes across one or moredimensions, comprises a plurality of matrices that corresponds tocollected data, and forms a plurality of vectors across at least one ofthe one or more dimensions; apply a plurality of Fourier Transforms onthe tensor to obtain a plurality of harmonic matrices in a Fourierdomain, wherein: the tensor includes a plurality of 2-dimensionalmatrices; and to apply the plurality of Fourier Transforms includes touse a plurality of processors to perform operations in parallel on datain the plurality of 2-dimensional matrices; perform singular valuedecompositions (SVDs) on the plurality of harmonic matrices to obtain aplurality of corresponding SVD results; reduce the plurality ofcorresponding SVD results, including selecting one or more dominantcomponents in the plurality of corresponding SVD results to obtain oneor more reduced SVD results; perform a plurality of Inverse FourierTransforms on the one or more reduced SVD results to obtain a de-noisedtensor that represents de-noised data; and output the de-noised tensorto be displayed or further processed.
 14. A computer program productembodied in a tangible non-transitory computer readable storage mediumand comprising computer instructions for: accessing input data includingmeasured and/or recorded data associated with a plurality of objects orentities, the measured and/or recorded data being used to construct atensor in a native domain, wherein the tensor: is a tensor of three ormore dimensions, represents interconnections of nodes across one or moredimensions, comprises a plurality of matrices that corresponds tocollected data, and forms a plurality of vectors across at least one ofthe one or more dimensions; applying a plurality of Fourier Transformson the tensor to obtain a plurality of harmonic matrices in a Fourierdomain, wherein the tensor includes a plurality of 2-dimensionalmatrices; and the applying of the plurality of Fourier Transformsincludes using a plurality of processors to perform operations inparallel on data in the plurality of 2-dimensional matrices; performingsingular value decompositions (SVDs) on the plurality of harmonicmatrices to obtain a plurality of corresponding SVD results; reducingthe plurality of corresponding SVD results, including selecting one ormore dominant components in the plurality of corresponding SVD resultsto obtain one or more reduced results; performing a plurality of InverseFourier Transforms on the one or more reduced results to obtain ade-noised tensor that represents de-noised data; and outputting thede-noised tensor to be displayed or further processed.
 15. A method,comprising: accessing input data including measured and/or recorded dataassociated with a plurality of objects or entities, the measured and/orrecorded data being used to construct a tensor comprising a plurality ofmatrices across one or more dimensions, wherein: the plurality ofmatrices represents nodes that have interconnections and that correspondto the plurality of objects or entities; and the tensor is a tensor ofthree or more dimensions and includes a plurality of 2-dimensionalmatrices; and clustering the tensor, including applying a tensor productfunction to the tensor to obtain a tensor graph that indicates changesin the interconnections of nodes across at least one of the one or moredimensions; wherein: the applying of the tensor product function to thetensor includes using a plurality of processors to perform operations inparallel on data in the plurality of 2-dimensional matrices; and thetensor graph obtained by applying the tensor product function to thetensor is equivalent to a result of convolving the tensor with itstranspose; and outputting at least a portion of the tensor graph to bedisplayed or further processed.
 16. The method of claim 15, wherein theapplying of the tensor product function includes convolving the tensorwith its transpose.
 17. The method of claim 15, wherein the applying ofthe tensor product function includes: performing a plurality of FastFourier Transforms (FFTs) on the tensor to obtain an FFT result;multiplying the FFT result with its transpose conjugate to obtain amultiplication result; and performing a plurality of Inverse FastFourier Transforms (IFFTs) on the multiplication result to obtain thetensor graph.
 18. The method of claim 15, further comprising identifyingone or more vertices using the tensor graph.
 19. The method of claim 15,further comprising de-noising the tensor graph.
 20. The method of claim15, further comprising de-noising the tensor graph, including: applyinga plurality of Fourier Transforms on the tensor graph to obtain aplurality of harmonic matrices; performing singular value decompositions(SVDs) on the plurality of harmonic matrices to obtain a plurality ofcorresponding SVD results; reducing the plurality of corresponding SVDresults, including selecting one or more dominant components in theplurality of corresponding SVD results to obtain one or more reducedresults; and performing a plurality of Inverse Fourier Transforms on theone or more reduced results to obtain a de-noised tensor.
 21. The methodof claim 15, wherein the accessing of the input data includes accessingthe tensor using an identifier, a pointer, a handle, or a reference to astorage or memory location of the tensor.
 22. The method of claim 15,wherein the applying of the tensor product function includes performingat least some operations in parallel on a plurality of processors.
 23. Asystem, comprising: a plurality of processors; and one or more memoriescoupled with the plurality of processors, wherein the one or morememories are configured to provide the plurality of processors withinstructions which when executed cause the plurality of processors to:access input data including measured and/or recorded data associatedwith a plurality of objects or entities, the measured and/or recordeddata being used to construct a tensor comprising a plurality of matricesacross one or more dimensions, wherein: the plurality of matricesrepresents nodes that have interconnections and that correspond to theplurality of objects or entities; and the tensor is a tensor of three ormore dimensions and includes a plurality of 2-dimensional matrices; andcluster the tensor, including to apply a tensor product function to thetensor to obtain a tensor graph that indicates changes in theinterconnections of nodes across at least one of the one or moredimensions; wherein: to apply the tensor product function to the tensorincludes to use a plurality of processors to perform operations inparallel on data in the plurality of 2-dimensional matrices; and thetensor graph obtained by applying the tensor product function to thetensor is equivalent to a result of convolving the tensor with itstranspose; and output at least a portion of the tensor graph to bedisplayed or further processed.
 24. A computer program product embodiedin a tangible non-transitory computer readable storage medium andcomprising computer instructions for: accessing input data includingmeasured and/or recorded data associated with a plurality of objects orentities, the measured and/or recorded data being used to construct atensor comprising a plurality of matrices across one or more dimensions,wherein: the plurality of matrices represents nodes that haveinterconnections and that correspond to the plurality of objects orentities; and the tensor is a tensor of three or more dimensions andincludes a plurality of 2-dimensional matrices; and clustering thetensor, including applying a tensor product function to the tensor toobtain a tensor graph that indicates changes in interconnections ofnodes across at least one of the one or more dimensions; wherein: theapplying of the tensor product function to the tensor includes using aplurality of processors to perform operations in parallel on data in theplurality of 2-dimensional matrices; and the tensor graph obtained byapplying the tensor product function to the tensor is equivalent to aresult of convolving the tensor with its transpose; and outputting atleast a portion of the tensor graph to be displayed or furtherprocessed.